Goto

Collaborating Authors

 digital human


Integrating Personality into Digital Humans: A Review of LLM-Driven Approaches for Virtual Reality

Brito, Iago Alves, Dollis, Julia Soares, Färber, Fernanda Bufon, Ribeiro, Pedro Schindler Freire Brasil, Sousa, Rafael Teixeira, Filho, Arlindo Rodrigues Galvão

arXiv.org Artificial Intelligence

The integration of large language models (LLMs) into virtual reality (VR) environments has opened new pathways for creating more immersive and interactive digital humans. By leveraging the generative capabilities of LLMs alongside multimodal outputs such as facial expressions and gestures, virtual agents can simulate human-like personalities and emotions, fostering richer and more engaging user experiences. This paper provides a comprehensive review of methods for enabling digital humans to adopt nuanced personality traits, exploring approaches such as zero-shot, few-shot, and fine-tuning. Additionally, it highlights the challenges of integrating LLM-driven personality traits into VR, including computational demands, latency issues, and the lack of standardized evaluation frameworks for multimodal interactions. By addressing these gaps, this work lays a foundation for advancing applications in education, therapy, and gaming, while fostering interdisciplinary collaboration to redefine human-computer interaction in VR.


Beyond Deep Fakes

Communications of the ACM

Within the next five years, the way we work, live, play, and learn will be changed by digital humans (chatbots and avatars with very realistic human faces). Digital humans are already gaining popularity as social media influencers, and they will soon evolve into digital sales assistants, fashion advisers, and personal shoppers able to model how customers will look and move in the latest ensembles. Digital humans will become central to the multibillion-dollar fashion industry, as social media is further integrated into the retail customer experience. Digital humans will also help in healthcare, enabling medical students and social workers to develop better interview skills for patients in sensitive clinical settings. They will allow people, especially those with mental health challenges, to rehearse for job interviews. They will help keep elderly people connected to their communities and respectfully monitored so they can remain in their homes longer. They will provide a human face for personalized advice, support, and training--and do it at scale. This has become possible with the advent of cost-effective, highly realistic, personalized interactive digital agents and avatars sporting high-fidelity facial simulations powered by advances in both real-time neural rendering (NR) and low-latency computing. NR refers to the use of machine-learning (ML) techniques to generate digital faces or face replacements in video.17 NR rose to prominence with the advent of so-called "deep fakes"--the replacement of someone's face in videos with an NR-generated face of remarkable realism. The term originates from the name of a Reddit user (/u/deepfakes), a ML engineer who posted the original deep fake auto-encoder. Often used for satire, deep fakes can be harmful, presenting novel ethical issues. The best-known examples involve deep fakes of celebrities, a form of face "hijacking" whereby publicly available videos of a person are used to train an ML program that overlays the source person's face onto existing video footage; this technique was originally used in pornographic material.


A Fusion Model: Towards a Virtual, Physical and Cognitive Integration and its Principles

Zhang, Hao Lan, Xue, Yun, Lu, Yifan, Lee, Sanghyuk

arXiv.org Artificial Intelligence

Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), digital twin, Metaverse and other related digital technologies have attracted much attention in recent years. These new emerging technologies are changing the world significantly. This research introduces a fusion model, i.e. Fusion Universe (FU), where the virtual, physical, and cognitive worlds are merged together. Therefore, it is crucial to establish a set of principles for the fusion model that is compatible with our physical universe laws and principles. This paper investigates several aspects that could affect immersive and interactive experience; and proposes the fundamental principles for Fusion Universe that can integrate physical and virtual world seamlessly.


D-ID unveils new chat API to enable face-to-face conversations with an AI digital human

#artificialintelligence

D-ID, the Israeli company leveraging artificial intelligence to create unique experiences like Deep Nostalgia, announced today that it's launching a new chat API to enable face-to-face conversations with an AI digital human. The announcement was timed to coincide with Mobile World Congress (MWC), which is taking place in Barcelona this week. The company is currently offering the API to enterprises for branding and customer experience purposes. The premise of the API is to provide a "human" interface for conversational AI. In a press release, D-ID said that with its new real-time streaming capabilities and its text-to-video technology, clients can integrate the power of large language models like GPT-3 and LaMDA to deploy interactive digital humans.


The stochastic digital human is now enrolling for in silico imaging trials -- Methods and tools for generating digital cohorts

Badano, A, Lago, M, Sizikova, E, Delfino, JG, Guan, S, Anastasio, MA, Sahiner, B

arXiv.org Artificial Intelligence

Randomized clinical trials, while often viewed as the highest evidentiary bar by which to judge the quality of a medical intervention, are far from perfect. In silico imaging trials are computational studies that seek to ascertain the performance of a medical device by collecting this information entirely via computer simulations. The benefits of in silico trials for evaluating new technology include significant resource and time savings, minimization of subject risk, the ability to study devices that are not achievable in the physical world, allow for the rapid and effective investigation of new technologies and ensure representation from all relevant subgroups. To conduct in silico trials, digital representations of humans are needed. We review the latest developments in methods and tools for obtaining digital humans for in silico imaging studies. First, we introduce terminology and a classification of digital human models. Second, we survey available methodologies for generating digital humans with healthy and diseased status, and examine briefly the role of augmentation methods. Finally, we discuss the trade-offs of four approaches for sampling digital cohorts and the associated potential for study bias with selecting specific patient distributions.


VR Applications in Healthcare & Medicine: Digital Humans

#artificialintelligence

We are seeing lots of developments involving the application of virtual reality (VR), extended reality, and augmented reality in healthcare and medicine. VR healthcare companies are applying these technologies to create virtual environments where patients and caregivers can participate in highly realistic and interactive experiences designed to simulate various healthcare scenarios and needs. Key application areas include education/training, treatment, and physical therapy/rehabilitation. This Advisor series examines the use of VR in healthcare and medicine. Part I covered how companies are using VR technologies to develop applications for healthcare educational and training purposes.


Perspective: The Metaverse Is Ushering in the Next Era of Computing

#artificialintelligence

The term meta, by its most modern definition can be described as self-referencing or self-reflective. In contemporary nomenclature, meta is often used as a standalone adjective. A "meta" name for a dog would be Dog or a meta movie – would be a movie about movies. And so, we have the metaverse. Another world for people and businesses to inhabit to conduct transactions and interact without the necessity of being fully, physically present.


AI is going to change UX research forever

#artificialintelligence

The rise of AI is creating a lot of buzz in almost every modern sector. While it remains unclear what we can expect from AI for designers, there have been recent developments that signify that something huge is going to happen. We may see some groundbreaking development in the way we handle interactions. Digital humans are becoming more present on the internet, and may revolutionize how we interact with the world around us. Our creativity can also expect a huge boost from AI.


Metaverse is a virtual reality wherein AI is the Growth Engine!

#artificialintelligence

AI is the growth engine of Metaverse. To fire the Metaverse engine, you need AI Fuel. I know you can't agree more! The idea of the metaverse was first coined by science fiction writer Neal Stephenson in the early 90s. It was eventually developed in parts by companies like Second Life, Decentraland, Microsoft, and most recently, Meta.


A digital human could be your next favorite celebrity--or financial advisor

MIT Technology Review

"Rising demand is driving the boom of digital humans," says Shiyan Li, head of the digital human and robotics business at Baidu, which created the digital model-actor, Gong. "In China alone, there are over 400 million ACGN (animation, comics, games, and novel) fans, and an enterprise market worth hundreds of billions of dollars centered on digital humans." And according to a company that tracks business registrations, Qichacha, China now has more than 280,000 enterprises that engage in digital human-related activities. The debut of Baidu's digital celebrity may not seem like much at first, as the concept of "virtual idols" has been around for years. For example, US virtual influencer Lil Miquela has been appearing alongside real human celebrities in online advertisements and TV commercials since 2016, gaining over three million Instagram followers.